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AN INVESTIGATION OF CORRELATION 


BETWEEN PILOT SCANNING BEHAVIOR AND WORKLOAD 
USING STEPWISE REGRESSION ANALYSIS 

Marvin C. Waller 
Langley Research Center 

SUMMARY 

An electro-optical device called an oculometer which tracks a subject's lookpoint as 
a time function has been used to collect data in a real-time simulation study of instrument 
landing system (ILS) approaches. The data describing the scanning behavior of a pilot 
during the instrument approaches have been analyzed by using a stepwise regression anal- 
ysis technique. A statistically significant correlation between pilot workload, as indicated 
by pilot ratings, and scanning behavior has been established. In addition, it was demon- 
strated that parameters derived from the scanning behavior data can be combined in a 
mathematical equation to provide a good representation of pilot workload. 

INTRODUCTION 

A program is underway at the NASA Langley Research Center (LRC) to develop a 
method of measuring pilot workload by use of an objective measure based on pilot scanning 
behavior. This technique would be particularly applicable to changes in workload induced 
by display changes or changes in information presented to the pilot. Traditional measures 
of workload have included subjective ratings such as Cooper-Harper ratings (ref. 1), and 
the use of sensors attached to the subject's body to make objective measures of heart rate, 
blood pressure, skin resistance, body temperature, etc. (ref. 2). Also, requiring the sub- 
ject to perform a secondary task has been used extensively in attempts to measure work- 
load (ref. 2) objectively. Some of these measurements could burden the pilot, possibly 
interfere psychologically or physiologically with his comfort or performance of the 
assigned task, and may even themselves add to the workload. 

The approach which is presently under investigation at Langley involves measuring 
the scanning behavior of the subject as a time function by use of an electro-optical device 
called an oculometer. The primary parameters which the oculometer measures as time 
functions are the x- and y-coordinates of the subject's lookpoint. These two output sig- 
nals are used to compute several variables which characterize the scanning behavior of 



the subject. They will be described in more detail later. An attempt is made to combine 
some of the derived scanning behavior variables into a linear equation to represent pilot 
rating, which is assumed to be a measure of workload. Reference 1 discusses the rela- 
tionship between pilot rating and workload. The process of finding such an equation 
establishes a correlation between pilot ratings and scanning behavior. 

To derive such an equation, a stepwise regression analysis program has been 
selected. This technique of analysis which will be outlined later in this discussion has 
at least two advantages. One is that a large number of variables may be considered for 
inclusion in the final result without forcing the user himself to select a subset. A second 
advantage is that it facilitates selection of independent variables which correlate to the 
residual after the effects of already selected variables have been taken into account. 
These advantages of the stepwise regression process, assuming a model is sought, pro- 
vide a convenient method for investigating the correlation between workload and scanning 
behavior. 

The present study investigates correlation between pilot ratings and scanning 
behavior during a simulated instrument approach study conducted on the Langley visual- 
motion simulator (VMS). The pilot ratings were supplied by a NASA test pilot and are 
generally representative of ratings given to these tasks by several NASA test pilots. 

Also reference 1 addresses the general reliability of pilot ratings provided by test pilots 
experienced in rating with the Cooper-Harper and similar scales. The scope of the work 
reported here is limited in that only four pilots participated in the tests. In addition, one 
specific part of the flight regime, an instrument landing system (ILS) approach, was con- 
sidered; thus, any results may not be generalized over the entire flight regime. Also, 
task difficulty is varied by manipulating the test setup in only a few of the possible ways. 

A general discussion of the data used in this study is presented in reference 3. 

SYMBOLS AND ABBREVIATIONS 

Values are given in both SI and U.S. Customary Units. The measurements and cal- 
culations were made in U.S. Customary Units. 

ith variable in regression equation 

Ix,Iy,Iz moments of inertia about X-, Y-, and Z-axis, respectively, kg-m^ (slug-ft^) 
IXZ>IXY>*YZ products of inertia, kg-m2 (slug-ft^) 
m,n number of rows and columns, respectively 
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X 

x i 

Y 

ALT 

AS 

CDC 

CL 

FD 

GSI 

HSI 

(I,J) 

LTC 
MDT ( ) 
NRP 
R 

Rg DF 
Rg F 
Rg MS 


matrix of unknown coefficients 

coefficient of ith scanning behavior variable 

response variable or pilot rating 

altimeter 

airspeed indicator 

Control Data Corporation 

clock 

flight director and total counts on flight director 
glide -slope indicator 
horizontal situation indicator 

probability of transition from instrument I to instrument J when I and J are 
replaced by CL, AS, FD, ALT, HSI, and VSI. Also, probability of transi- 
tion from location I to location J in the flight director; here I and J assume 
the values 1 to 9. 

lost track count 

mean dwell time on instrument in parentheses, sec 

number of 1/32-sec records processed, run length 

multiple correlation coefficients 

regression degrees of freedom 

regression F- ratio 

regression mean square 
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Rg SS regression sum of squares 

Rs DF residual degrees of freedom 

Rs MS residual mean square 

Rs SS residual sum of squares 

SDT ( ) standard deviation of dwell time on instrument in parentheses 

S E standard error of regression 

TTOI total time on instrument, 1/32 -sec counts 

TTT total tracking time, sec 

VSI vertical speed indicator 

SIMULATION HARDWARE 

The Langley visual- motion simulator (VMS) was used in the fixed-base mode. A 
photograph of the interior of the VMS cockpit is presented in figure 1. The cockpit equip- 
ment included an instrument panel with instrumentation typical of that found in operational 
transports. It is, perhaps, of some significance that no engine instruments were visible 
to the pilot because the mounting of the optical head, as shown in figure 1, blocked their 
view. However, jet engine noise was simulated and presented to the pilot by speakers 
located in the rear of the cockpit. 

During the simulation only the six instruments in the basic T-arrangement pre- 
sented in the photograph of figure 2 were used by the pilot. These instruments are the 
clock, the airspeed indicator, the flight director or attitude direction indicator (depend- 
ing on the configuration of the instrument), the altimeter, the horizontal situation indi- 
cator, and the vertical speed indicator. The flight director used in this study is depicted 
in figure 3. The command bars of the flight director were programed according to speci- 
fications initially supplied by the Boeing Company for a terminal configured vehicle simu- 
lation program. Comments from the pilots indicated that this flight director system had 
some deficiencies under the heavy turbulence condition studied. Such conditions, however, 
were probably pushing the use of the system beyond its design criterion. Also, the flight 
director contained a speed command indicator (speed bug). This indicator with gradua- 
tions at 5- and 10-knot increments indicated deviation from the nominal 120 -knot airspeed. 
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Flight director 



Clock Airspeed 


Altimeter 


Vertical speed 
indicator 


L-76-118 


Horizontal situation indicator 


Figure 2.- Basic T-arrangement of the six instruments. 
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(b) Gridding used to describe lookpoint in flight director. 
Figure 3.- Flight director. 
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The speed command is not universally available in airline and military flight director 
instruments. Pilot comments, in addition to those about the flight director, indicated 
that the airspeed indicator was not very satisfactory because it could not be read as 
fast as they would have liked on account of the size of the numbering on its face. 

The simulator cockpit was also equipped with a control column and wheel, a thumb - 
activated pitch and roll trim switch, rudder pedals, and a right -hand -operated engine 
throttle control. No out -of -the -window visual scene was presented. 

The simulator was controlled from the Langley real-time digital computer system 
(CDC 6600) in which the equations of motion, aircraft dynamics and control system, dis- 
play instrument dynamics, and routines for recording data on magnetic tape were all 
programed. The computing cycle was 1/32 of a second and data were recorded on the 
magnetic tape during each cycle. 

DESCRIPTION OF OCULOMETER 

The oculometer is an electro-optical device built by the Honeywell Radiation 
Corporation (NASA contracts NASw-1159 and NAS 12 -531). The basic principle of 
operation involves illuminating the subject’s eye with infrared radiation and monitoring 
the reflected radiation with an infrared- sensitive television (TV) camera and using an 
associated mini-computer for processing the signal. More detailed discussions of the 
operating principle of the oculometer are available in references 3 and 4. The tracking 
accuracy of the oculometer is 0.5° radius and head movement in a cubic foot volume is 
allowed. 

The electro-optical package of the oculometer is the only part of the system which 
has to be located in the cockpit. Its location for the present study can be seen in fig- 
ure 1. It must be mounted so that its beam can be directed at the subject's eye (one eye 
is tracked). This is the primary restriction in using the oculometer in the simulator 
environment. It is a remote tracking system and thus requires no attachments to the 
subject and hence should not impact the performance of the subject. 

In addition to recording the lookpoint data on magnetic tape, a TV camera was 
mounted in the cockpit to view the instrument panel over the right shoulder of the sub- 
ject. This video image of instruments with a moving dot (generated by the associated 
electronics of the oculometer) superimposed to represent the subject’s lookpoint is 
recorded on video tape. The video tape data were used in analyzing the results to pro- 
vide confirmation of the trends pointed out by the regression analysis technique. 


7 



THE SIMULATED AIRCRAFT 


The aircraft simulated in this study is a large transport. Some of the relevant 
characteristics of the aircraft are 


Weight, N (lb) . . . . 

Mass, kg (slugs) 

Moments of inertia, kg-m 2 (slug-ft 2 ): 

ix 

Iy 

l z • • • 

Products of inertia, kg-m 2 (slug-ft 2 ): 

l XZ 

r XY 

l YZ 

Wing span, m (ft) 

Mean aerodynamic chord, m (ft) . . 
Wing area, m2 (ft 2 ) 


. . . 40 003.4 (90 000) 

. . . 40 823.3 (2797.28) 

5.08 x10 s (3.75 x 10 s ) 
11.86 x 10 s (8.75 x 10 s ) 
16.27 x 10 s (12.0 x10 s ) 

6.51 x 10 4 (4.80 x 10 6 ) 

0.0 ( 0 . 0 ) 

0.0 ( 0 . 0 ) 

28.35 (93) 

3.41 (11.20) 

91.05 (980.00) 


Control surface deflection limits are 


Rudder deflection, deg ±24 

Elevator deflection, deg ±21 

Aileron deflection, deg ±20 


SIMULATION TEST CONDITIONS 


The test design used in the present study consisted of instrument approaches made 
in the six conditions listed in table I. Changes in turbulence level and display modifica- 
tions were used to vary workload. The display changes consisted of conditions with and 
without the flight director glide slope and localizer command bars. Also the speed com- 
mand was covered in one test condition. 

TABLE I.- TEST CONDITIONS 


Condition 

Initial offset (horizontal) 

Display 

Cross wind, 
deg 

Turbulence 

Pilot rating 

m 

ft 

I 

0 

0 

No speed command 

0 

None 

3.0 

H 

0 

0 

Normal 

0 

None 

2.5 

III 

152.4 

500 

No command bars 

0 

None 

4.0 

IV 

152.4 


Normal 

45 

Moderate 

3.5 

V 

152.4 

500 

No command bars 

45 

Heavy 

7.0 

VI 

152.4 

500 

Normal 

45 

Heavy 

5.0 
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Condition II is considered the nominal condition. The speed command on the flight 
director was covered in condition I to allow assessment of the effects of the speed com- 
mand on the pilot's scanning behavior. In conditions III and V, the pitch and roll com- 
mand bars in the flight director were inoperative. These conditions allowed evaluation 
of scanning behavior and performance when basic instruments, without command infor- 
mation available, were used. Also conditions IV, V, and VI have turbulence and a 15-knot 
(7.72-m/s) cross wind added to allow the effects of these disturbances to be determined. 


The instrument approaches were started 10 058 m (33 000 ft) from the runway 
threshold at an altitude of 487.68 m (1600 ft). Nominal airspeed for the approach was 
222.2 km/hr (120 knots). The cross wind entered the simulation at 6705.6 m (22 000 ft) 
from the runway threshold. The turbulence was simulated by the Dryden turbulence 
model presented in reference 5. The flight was along a 3° glide path and required about 
2^ minutes for completion. 


Four pilots were used as subjects in the tests. Three of these were U.S. Air Force 
pilots current in a large transport (C-130) and one was a NASA test pilot current in the 
Boeing 737 as well as a wide variety of other aircraft. 


STEPWISE REGRESSION ANALYSIS 


Regression analysis is a method of identifying the parameters of an overdetermined 
system of equations. In general, the term will include such techniques as least squares, 
maximum likelihood, minimum variance, and several other classical means of estimating 
the parameters of such a system. The particular estimation technique used in the pres- 
ent study is least squares. A complete discussion of this process can be found in many 
textbooks on the subject of regression analysis. Reference 6 is a good example, and the 
reader is referred there for a detailed discussion of the least-squares process and 
regression analysis. 

As a brief overview of the subject, however, assume that one wishes to solve a sys- 
tem of overdetermined linear equations represented in matrix notation by 

Y = AX + e 

where Y is an m by 1 column vector, A is an m by n matrix, X is an n by 1 vector, 
and e is an m by 1 vector of experimental errors. The simple least-squares solution 
for X which minimizes the sum of the squares of the experimental error e, when appro- 
priate assumptions (ref. 6) are used, is 

x = (a t a)’ 1 a t y (1) 

where A^ is the transpose matrix. 
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In application then, when given m measurements of Y, the response, and corre- 
sponding observations of the n elements in the rows of the A matrix, . m > n, equa- 
tion (1) will provide the simple least- squares solution to the overdetermined system. 

With no rigor intended in the discussion, this solution guarantees that the sum of the 
squares of the residuals or errors is minimized. 

It was assumed in the previous discussion of regression analysis that a linear equa- 
tion with unknown coefficients had been selected to represent the response Y in terms 
of the independent variables of the A array before starting the process. In the subject 
application, however, choosing a set of independent variables is a part of the problem and 
is accomplished by the stepwise regression process. Stepwise regression analysis begins 
with no model; thus, it is assumed that the model developed will be a linear combination of 
some subset of the independent variables considered, and a set of m observations of the 
response and the independent variables. It proceeds to find the best (in some sense such 
as least squares) set of variables to include in the linear model. This process is applica- 
ble when several independent variables have been measured because they are thought to be 
correlated to the response and yet the researcher does not know whether all should be 
included in the model or some subset. 

The stepwise regression analysis proceeds by entering into the model first the inde- 
pendent variable which is most highly correlated to the response. It enters next the inde- 
pendent variable which is most highly correlated to the response when given the effects of 
the previously included variable. This is the same as choosing next the variable most 
highly correlated with the residual. A second criterion in addition to correlation of a 
variable to the response is imposed. The significance of the variable under considera- 
tion is tested against the F-distribution; therefore, its F-ratio must exceed a designated 
critical value. Also each time a new variable is entered into the model, the variables 
already in the model are each checked to ascertain that their F-ratios still exceed a des- 
ignated critical value. If this is not the case for any variable, it is removed from the 
regression equation. Imposing this second criterion insures statistical significance of 
the result. The process is terminated when no additional independent variables can be 
added to the model. The output from the stepwise regression is then a set of independent 
variables which can be combined in a linear model to represent the response. Also, the 
coefficients (X vector of eq. (1)) are computed. Statistics to allow an analysis of vari- 
ance or to evaluate goodness of fit of the resulting equation are included in the output of 
the particular computer program used. (See ref. 7.) 
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APPLICATION OF STEPWISE REGRESSION ANALYSIS TO 
APPROXIMATE WORKLOAD 


The data from the oculometer were reduced first to an instrument-to- instrument 
transitions probability matrix. This matrix was computed by counting the number of 
1/32 -second intervals the pilot was found looking at each instrument location. A diag- 
onal term of this matrix is incremented each interval that the subject is found looking at 
the same instrument as during the previous interval. An appropriate off-diagonal term 
is incremented when the subject is found to have changed instruments. The counts in this 
matrix are normalized to give probabilities by dividing each element by the total number 
of time intervals that the oculometer tracked the subject. Table II presents an example 
of such a 6 by 6 transition probability matrix for the six instruments (fig. 2) included in 
the data analysis of this study. The elements of this matrix are among the independent 
variables considered in the stepwise regression and are represented by terms such as 
(HSI,FD) to designate the probability of transition from the horizontal situation indicator 
(HSI) to the flight director (FD). 

TABLE II.- EXAMPLE OF A TRANSITION PROBABILITY MATRIX 


FROM 

CL 

AS 

TO 

FD 

ALT 

HSI 

VSI 

CL 

0.0001 

0.0000 

0.0000 

0.0000 

0.0000 

0.0000 

AS 

.0000 

.0053 

.0003 

.0000 

.0000 

.0000 

FD 

.0000 

.0003 

.9020 

.0013 

.0009 

.0014 

ALT 

.0000 

.0000 

.0012 

.0133 

.0000 

.0000 

HSI 

.0000 

.0000 

.0008 

.0000 

.0108 

.0001 

VSI 

.0000 

.0000 

.0016 

.0000 

.0000 

.0203 


In a similar analysis a transition probability matrix for the scanning behavior of 
the subject within the flight director was derived. The flight director instrument is 
divided into a 3 by 3 array of rectangles as illustrated in figure 3(b). These rectangles 
are numbered one (1) to nine (9) as indicated in the figure. To derive a transition prob- 
ability matrix for the flight director, the entries in the transition matrix derived by fol- 
lowing this scheme are divided by the total number of counts the subject looked at the 
flight director. The elements of the matrix are also included in the stepwise regression 
process and are represented in the discussion of the results by symbols such as (1,2) to 
mean number of transitions from the square numbered 1 to that numbered 2 (fig. 3), etc. 
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The other variables which were considered as candidate independent variables in 
the stepwise regression analysis included mean and standard deviation of the dwell time 
(or duration of fixations) on each of the six instruments and on each of the nine squares 
dividing the flight director. Also, variables which were tallies on number of records 
processed, total time the oculometer tracked, etc., were included. 

In addition to each variable mentioned, its square was also included. However, no 
product or interaction terms were considered. Measurements of these variables made 
up the A matrix in equation (1). The response variable Y was the pilot rating for 
each of the six conditions. These were the Cooper- Harper ratings assigned to each of 
the six test conditions by a NASA test pilot. Table I which lists the test conditions also 
includes the pilot rating given to each condition. 

The results of the stepwise regression analysis will be an equation of the form 

Y = Y. a i x i + x o 

where the a^'s are the scanning behavior variables included in the equation after the 
stepwise regression analysis process has been completed. The x^'s are the corre- 
sponding coefficients and x Q is a constant. 

The application of stepwise regression analysis in the present study included an 
algorithm which was constructed to overcome difficulties associated with sorting a large 
number of candidate independent variables. Two hundred and ten data runs were availa- 
ble for analysis. One hundred and fifty candidate independent variables plus the square 
of each were selected for the stepwise regression analysis process. Analysis of the 
programing requirements to handle this amount of data indicated that a large amount of 
computer memory is necessary to execute stepwise regression analysis directly. The 
assumption is made that the process would stop before the number of variables included 
in the equation approached the number of measurements. An alternate processing pro- 
cedure was sought. The variables were randomly divided into eight groups. The step- 
wise regression process was performed on each group separately. Since it was possible 
that some variables could be zero all or most of the time, it was arbitrarily decided to 
eliminate from the analysis any variable which was found to be zero in 95 percent of the 
runs. Therefore, group size varied in the analysis. The variables entering the equation 
resulting from the processing of each of the eight groups were next combined in a final 
stepwise regression process. 

An algorithm such as the one discussed requires an investigation of the stability 
or repeatability of the results. This is true partly because of the existence of correla- 
tion between the candidate variables. Such correlation could make variable grouping an 


12 



important factor influencing the results. It could mean that there is more than one set 
of variables which could form an equation representing the response variable equally 
well. 


RESULTS AND DISCUSSION 

A typical example of the results of the stepwise regression program follows. The 
purpose for presenting the example is to provide the reader with enough of the details 
of one attempt at a solution to the problem to see how the stepwise analysis program 
functioned. 

The variables entering the program were first randomly subdivided into eight (8) 
groups. When a variable was assigned to a group, its squared value was also assigned to 
the same group. Table III lists the variables entering each group in a separate column. 


TABLE m.- AN EXAMPLE OF'RESULTS OF STEPWISE REGRESSION PROCESS USED 


Statistical 

measures 

Group 

FINAL 

1 

2 

3 

4 

■ 5 

6 

7 

8 


(HSI, VSI) 

SDT (2) 

SDT (6) 

(9,8) 

LTC 

SDT (9) 

(5,9) 

TTT 



a (6,2) 

a (ALT,HSI) 

(2,3) 

(5,3) 

(6,6) 

(ALT, ALT) 

a (8 ,7) 

(7,4) 



b MDT (VSI) 

(4,5) 

(5,6) 

a (7,8) 

b FD 

b (4,2) 

(1,4) 

(7,7) 

(FD,HSI) 


(2.8) 

c (l,2) 

(FD.ALT) 

(8,6) 

(9,5) 

(6,5) 

b (2,4) 

a (8,8) 

(TTOI) 2 


a MDT (HSI) 

a (ALT,FD) 

(9,7) 

a (2,5) 

NRP 

(9,9) 

SDT (1) 

Emm 

(5,2)2 


MDT (6) 

SDT (4) 

(4,7) 

c ( VSI, VSI) 

a SDT (VSI) 

C (HSI,HSI) 

(2,1) 

EH 

(3,2) 2 


(5,4) 

C MDT (FD) 

a (5,8) 

a (VSI,ALT) 

MDT (ALT) 

(6,3) 

a (AS,AS) 

b (3,6) 

(4,8) 


(HSI, ALT) 

MDT (3) 

(9,6) 

MDT (4) 

(VSI, HSI) 

(8,9) 

SDT (9) 

(7,5) 

(6,2) 


a (FD,AS) 

MDT (5) 

a (M) 

(6,8) 

b MDT (9) 

(ALT, VSI) 

b (FD,FD) 

a (4 ,8) 

(5,1) 


C SDT (8) 

SDT (ALT) 

a (FD,HSI) 

(5,5) 

a (HSI,FD) 

b TTOI 

MDT (1) 

(3,5) 



c (8,5) 


a MDT (7) 

(4,1) 

, (2,6)’ 

(3,3) 

b (5,2) 

a (AS, FD) 



(7,9) 


(2,2) 

MDT (AS) 

b SDT (AS) 

a SDT (3) 

(FD,VSI) 

a (5,l) 



(VSI.FD) 



MDT (2) 

MDT (2) 


a SDT (5) 

(8,4) 






(1,5) 

b (3,2) 


SDT (FD) 

(5,7) 


Rs DF 

192 

194 

196 

195 

194 

195 

194 

195 

193 

Rs SS 

240 

307 

213 

314 

168 

172 

190 

298 

120 

Rs MS 

1.2 5 

1.58 

1.08 

1.61 

0.86 

0.88 

0.98 

1.53 

0.62 

Rg DF 

9 

7 

5 

6 

7 

6 

7 

6 

8 

Rg SS 

3885 

3818 

3912 

3810 

3957 

3953 

3934 

3827 

4005 

Rg MS 

431 

545 

782 

635 

565 

658 

562 

638 

500 

Rg F 

343 

344 

717 

393 

651 

743 

571 

416 

804 

S E 

1.18 

1.25 

1.04 

1.27 

0.93 

0.94 

0.99 

1.24 

0.78 

R 

0.97 

0.96 

0.97 

0.96 

0.97 

0.97 

0.97 

0.96 

0.99 


a Variable placed in equation. 
b Square placed in equation. 
c Both variable and square included. 
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The column headed "FINAL" lists the variables entering the regression equation when the 
stepwise regression process is carried out on the combined results from the eight prelim- 
inary groups. Although not listed in the table, a constant is included in each group. Also 
the following statistics about the results of each regression step are listed in each column: 

Rs DF . residual degrees of freedom, number of observations minus Rg DF 

Rs SS residual sum of squares, sum of squared values of differences between rating 
provided by test pilot and that predicted by model 

Rs MS residual mean square, Rs SS/Rs DF 

Rg DF regression degrees of freedom, number of independent variables included in 
predicting equations 

Rg SS regression sum of squares, sum of squared values of predicted response 

variable 

Rg MS regression mean square, Rg SS/Rg DF 

Rg F regression F-ratio, Rg MS/Rs MS 

S E standard error of regression, \jRg MS 

R multiple correlation coefficient of regression, 

The program also computes the correlation coefficients of the variables a^ considered 
in a particular group. These are available for printing. 

These statistical measures are discussed in detail in most texts on regression anal- 
ysis. (See, for example, ref. 6.) For a good estimating equation one would, in general, 
want the following (least- squares case): 

Rs SS minimum 

Rs Ms minimum 

Rg SS maximum 

Rg MS maximum 

Rg F maximum 

S E minimum 

R maximum, approaching 1.0 
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These statistics are interrelated and forcing one to approach the desired value usually 
causes the others to be incremented in the desired direction. Again, these comments 
are not meant to be a rigorous discussion but are intended to present an overall view of 
the analysis pursued here. The parameters also allow analysis of variance tests on the 
statistical significance of various estimating equations. 

The final estimating equation which resulted for this particular example is 

Y = 5.771 - (l.384 x 10- 7 )(TTOI) 2 + 229.2 (FD,HSI) + 669.2(5,2)2 
- 39 880.0(3,2) 2 +490.8(4,8) + 637.0(6,2) + 509.6(5,1) 

Figure 4 presents a plot of the average and standard deviation of the pilot rating 
estimated by the preceding equation compared with the actual pilot rating supplied by the 
NASA test pilot. 



0 1 2 3 4 5 6 7 

Actual pilot rating 

Figure 4.- Example of regression results. 
Standard error, 0.8. 
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Table IV presents the average value over all data runs used in this example for the 
variables which entered the resulting equation for estimating pilot rating. This might 
offer some clues as to the role of each variable in the resulting equation. 

TABLE IV.- VARIABLES REMAINING IN REGRESSION 
[Entries in tables are average data values] 


Condition 


Variable 


Pilot rating 

(dependent variable, Y) . . . 

Constant 

Square of number of counts 

on instruments 

Transition probability from 

FD to HSI 

Transition probability from 

block 5 to 1 

Transition probability from 

block 6 to 2 

Transition probability from 

block 4 to 8 

Square of transition prob- 
ability from block 5 to 2 ... 
Square of transition prob- 



V 

VI 

7.0 

5.0 

(3798)2 

(4987)2 

0.0074 

0.0037 

0.0003 

0.0001 

0.0003 

0.0000 

0.0005 

0.0003 

(0.0130)2 

CM 

o 

o 

o 

(0.0006)2 

(0.0001)2 


One apparent advantage in using stepwise regression analysis is that it can point 
out subtle correlations which might not have been considered in more conventional type 
analyses. In the example considered here the constant term entered the estimating equa- 
tion first. Then a correction term involving the total time spent on instruments and one 
involving the number of transitions to the HSI were entered. These might have been an 
obvious inclusion to an equation by considering the correlation matrix or just looking at 
the data. Some of the other terms which describe specifically how the subject used the 
flight director from condition to condition might well have easily been overlooked in a 
more conventional approach, particularly since these are the lower magnitude parameters 
in the flight director transition probability matrix. 
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Although looking at the average data presented in table IV does confirm that there 
is correlation between the test conditions and variation of all the parameters included in 
the resulting equation, the reason for this correlation in some cases might be a subject 
of speculation. The inclusion of frequency of transition to roll information from the 
flight director center (5,2)2 i s no t surprising. Transitions from the speed command to 
the localizer information (4,8) seem correlated to both the format of the flight director 
(absence of command bars) and the presence of a high turbulence level. The reason for 
its correlation with the latter is not clear. The reason for correlation of pilot rating and 
transitions from block 3 to roll information is not obvious. This transition in lookpoint 
is present in each condition tested to some degree. Reviewing several video tapes indi- 
cates that this behavior might be associated with the return path from the altimeter to 
the roll pointer on the flight director during the final phase of each run. Transitions 
from the glide -slope indicator (GSI) to the roll pointer (6,2) show up significantly only 
when the command bars or the flight director are not active. The reason for the pres- 
ence of transitions from center of the flight director to block 1 (5,1) is not obvious. It 
appears from looking at video tapes that this scanning behavior is associated with the 
amount of sequential and vigilant monitoring of the roll pointer and speed command. It 
appears that block 1 might be used as a good location for the pilot to see simultaneously 
the speed command and the roll pointer and return easily to the information in the cen- 
ter of the flight director. It is emphasized that these explanations of the reasons for 
the measured correlation are to a large degree educated guesses and may prove erro- 
neous as the phenomena involved become better understood. 

To investigate the stability or repeatability of the results of the processing used 
here, a routine was added to the computer program which grouped the variables entering 
the process randomly each time the program was run. Over several computer runs, the 
variables remaining in the regression were compared. 

It was observed that the results varied considerably from run to run. Table V pre- 
sents the results from executing the program several times. It was found, however, that 
several of the variables (or their squares) that are free to enter the results did so almost 
invariably: (1) a constant, (2) total time spent on the instruments, (3) number of transi- 
tions from the flight director to the HSI, (4) number of transition from the glide -slope indi- 
cator to the roll information in the flight director (6,2), and (5) number of transitions from 
the center of the flight director to location 1 in the flight director (5,1). 
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TABLE V.- RESULTS OF 10 EXECUTIONS OF STEPWISE REGRESSION ANALYSIS PROGRAM 
WITH VARIABLES RANDOMLY GROUPED 


Parameter 

a l 

a 2 

3 

a 4 

a 5 

6 

7 

8 

9 

10 

Total 

Constant 

V 

y 

y 

y 



y 

y 

y 

y 

10 

TTOI 

y 

y 


y 







3 

(FD.HSI) 

y 

y 

y 

y 

y 

y 

y 

y 

y 

y 

10 

(6,2) 2 

y 



y 

y 


y 

y 



6 

(4,8) 

y 

y 

y 

y 

y 



y 


y 

7 

(1,2) 

y 










1 

SDT (3) 

y 

y 


y 







3 

(2,5) 2 


y 









1 

(5,1) 


y 

y 

y 

y 

y 

y 

y 


y 

8 

(1,5)2 


y 




y 





2 

(5, l) 2 


y 




y 

y 




3 

(TTOI) 2 

(5,2)2 



y 

y 


y 

y 

y 


y 

y 

y 

M 

(2,3) 



y 








1 

(6,2) 



y 






y 

y 

3 

(3,5)2 




y 







1 

(2,3)2 





y 



y 

y 


3 

(FD) 2 





y 



y 



2 

MDT (VSI) 2 





y 



y 



2 

MDT (3) 





y 






1 

(AS, AS) 





y 



y 



2 

SDT (HSI) 





y 



y 



2 

(2,2) 

(3,2) 2 





y 


y 


y 

y 

H 

(5,3)2 






y 





1 

MDT (4) 2 







y 


y 


2 

(7,7)2 



- 




y 




1 

(4,7) 2 









y 


1 

(8,6) 









y 


1 

(FD) 









y 


1 

(6,3)2 









y 


1 

(3,3) 









y 


1 

(FD,AS) 









y 


1 

SDT (8)2 









y 


1 

(5,8) 









y 


1 

Rs DF 

194 

192 

193 

193 

189 

194 

192 

191 

186 

193 


Rs SS 

127 

125 

123 

121 

122 

130 

114 

123 

103 

120 


Rs MS 

0.65 

0.65 

0.64 

0.62 

0.64 

0.67 

0.59 

0.64 

0.57 

0.62 


Rg DF 

6 

8 

8 

7 

ii 

6 

9 

10 

15 

8 


Rg SS 

356 

3 58 

4001 

362 

361 

353 

4011 

4001 

4022 

4005 


Rg MS 

59 

44 

500 

51 

32 

58 

445 

400 

268 

500' 


Rg F 

90 

68 

780 

82 

50 

87 

746 

617 

481 

804 


S E 

0.81 

0.8 

0.80 

0.79 

0.80 

0.82 

0.77 

0.80 

0.74 

0.79 


R 

0.85 

0.86 

0.98 

0.86 

0.86 

0.85 

0.92 

0.98 

0.98 

0.99 


Intercept 

9.0 

8.6 

5.95 

8.6 

6.09 

6.08 

6.01 

5.9 

6.62 

5.77 



intercept (Constant) is forced into the regression results. Statistics account for role of the 
free variables only. 


















CONCLUDING REMARKS 

Stepwise regression analysis has been used in the present study to investigate 
the hypothesis that there is a correlation between workload measured in terms of a 
Cooper -Harper rating and pilot scanning behavior. This study has demonstrated that 
such a correlation does exist and that, in fact, scanning behavior parameters measured 
by an oculometer can be used to predict the pilot rating with standard error of 0.8 unit 
on the Cooper -Harper scale which ranges from 1 to 10. 

It was found that five parameters free to enter the regression results (or their 
squared value) did so almost invariably regardless of the manner in which the terms 
were grouped in the analysis. These parameters are 

(1) a constant, 

(2) total time spent on the instruments, 

(3) number of transitions from the flight director to the horizontal situation 

indicator (HSI), 

(4) number of transitions from the glide -slope indicator to the roll information in 

the flight director, and 

(5) number of transitions from the center of the flight director to location 1 in the 

flight director. 

One specific goal of the research conducted in this area is to establish a model for 
computing a pilot rating from oculometer measurements. The present study has demon- 
strated the feasibility of this goal by deriving example equations which can be used to 
compute pilot ratings for the six conditions of the present study. However, those exam- 
ple equations found here are not necessarily descriptive of the underlying principles 
involved in assigning pilot ratings to the tasks. They are models only in the sense that 
they can be used to represent the response as a mathematical function of the oculometer 
variables. Also, no claim is made of validity of these equations for predicting pilot rating 
under conditions different from the ones investigated in the present study. Nevertheless, 
this study has demonstrated the feasibility of approaching the more general problem. 

It is also pointed out that for purposes of the present study, consideration was not 
given to including in the analysis all oculometer variables which might correlate to work- 
load. This was not felt to be particularly necessary to establish whether a correlation 
exists between scanning behavior and workload measured in terms of pilot rating. It is 
believed, however, that developing a more general model should include such a step. 

Langley Research Center 

National Aeronautics and Space Administration 
Hampton, Va. 23665 
January 14, 1976 
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